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We use the work done on and the heat removed from a system to maintain it in a 
nonequilibrium steady state for a thermodynamic-like description of such a system 
as well as of its fluctuations. Based on a generalized Onsager-Machlup theory for 
nonequilibrium steady states we indicate two ambiguities, not present in an equi- 
librium state, in defining such work and heat: one due to a non-uniqueness of time- 
reversal procedures and another due to multiple possibilities to separate heat into 
work and an energy difference in nonequilibrium steady states. As a consequence, 
for such systems, the work and heat satisfy multiple versions of the first and sec- 
ond laws of thermodynamics as well as of their fluctuation theorems. Unique laws 
and relations appear only to be obtainable for concretely defined systems, using 
physical arguments to choose the relevant physical quantities. This is illustrated on 
a number of systems, including a Brownian particle in an electric field, a driven 
torsion pendulum, electric circuits and an energy transfer driven by a temperature 
difference. 



1 Introduction 



Of all steady states of systems, the equilibrium state is by far the most studied. First, a ther- 
modynamic description has been developed involving the work done by or on the system or the 
heat produced by or removed from the system. This leads to the first law of thermodynamics, 
i.e. the law of energy conservation, while the introduction of entropy leads to the second law of 
thermodynamics, i.e. that entropy changes in a closed system have to be non-negative. 

A generalization to systems in nonequilibrium steady states (NESS) has been made as a special 
case of the general theory of thermodynamics of irreversible processes (irreversible thermody- 
namics) for systems in local, i.e. near equilibrium [[T]|. This theory has in turn been enlarged 
to an extended irreversible thermodynamics where in addition to the usual local quantities 
(local mass density, local velocity and local energy density) also the corresponding irreversible 
thermodynamic currents of mass, momentum and energy (or heat) are taken into account for a 
description of the system. Also in that context NESSs can be considered. NESSs, as well as 
their fluctuations, have also be considered in hydrodynamics (31. 
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The major difference between all these thermodynamic theories of NESSs and the attempt pro- 
posed here to describe NESSs, is that all these theories are ultimately based on a direct general- 
ization of equilibrium thermodynamics and, in particular, the use of the same concepts of work 
and heat as in equilibrium. To the contrary, the theory developed here introduces fundamentally 
different definitions of work and heat, associated with a NESS, rather than those used in the 
above equilibrium thermodynamic based theories. In fact, we propose a thermodynamic-like 
description of systems in NESSs by defining the work associated with such a system as the 
work that has to be done on the system to maintain it in its NESS and prevent it from decay- 
ing to an equilibrium state. Similarly we define the heat associated with such a system as the 
heat that has to be removed from the system to eliminate the heat produced by the irreversible 
(nonequilibrium) processes which take place in such systems. 

We develop this theory for NESS using a generalization of the classical path integral theory of 
Onsager and Machlup flU for fluctuations in the equilibrium state to NESSs, used already by 
us in two previous papers for a specific model lISlO. An introduction to this theory, relevant 
for this paper, can be found in the first paper [[5]|. The present paper attempts to present the 
general structure, applied to a variety of models, for discussing the NESS as a generalization of 
the equilibrium state and to exhibit the conceptual differences between these two steady states 
of a system and their fluctuations. 

In fact, contrary to the NESS, the equilibrium state is an absolutely stable state, which maintains 
itself without the necessity of any work to be done on it, nor of the removal of any spontaneously 
produced heat, since this heat vanishes on average in an equilibrium state. For that reason 
the Onsager-Machlup theory of fluctuations of the equilibrium state, does not consider any 
work done on the system and only considers the entropy production rate associated with the 
fluctuations in the equilibrium state. The absence of any work allows the formulation of a theory 
of fluctuations in the equilibrium state, based on the entropy production associated with these 
fluctuations, alone. Therefore the Onsager-Machlup theory does not contain any equilibrium 
thermodynamic feature. 

This is completely different for a NESS, where the presence of external "forces" (characterized 
by appropriate nonequilibrium parameters), keeps the system permanently out of equilibrium, 
requiring "actions" involving work and heat to maintain this system in a NESS and prevent 
it from decaying to the absolutely stable equilibrium state. Then a generalization of the two 
laws of equilibrium thermodynamics is possible, which require, however, NESS adapted def- 
initions of work and heat, which are, together with the internal energy, the ingredients of a 
thermodynamic-like formulation of the NESS. 

The generalization of the Onsager-Machlup theory for fluctuations in the equilibrium state to 
one for NESS, turned out to be non-trivial. This, since such a generalization involves in general 
ambiguities, i.e. multiple a priori possible choices for the work, heat and energy and their fluc- 
tuations in a NESS. The Onsager-Machlup theory for the equilibrium state provides us though 
with a starting point to deal with these problems and to obtain a physically unique description 
of the thermodynamic laws and the fluctuations of a NESS, at least for the specific models 
considered in this paper. 
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Implementation of the above outlined program is based on Onsager and Machlup's path inte- 
gral method. This involves, not only the above mentioned new definitions of the NESS adapted 
thermodynamic-like quantities of work and heat, but also new definitions of forward and corre- 
sponding backward paths in time because of the presence of external nonequilibrium parameters 
in the NESS. The main difficulty and the origin of these ambiguities arising then is that an ap- 
propriate choice of a backward path for a given forward path is not unique and depends on the 
nature of the system in the NESS. In addition, there is an intrinsic ambiguity because work and 
energy differences can only be defined up to a common quantity. It appears at present that these 
ambiguities can only be resolved on physical grounds for specific concrete models. 

The contents of this paper are organized in three parts as follows. After the introduction in this 
sectionfH we introduce in Sec. [2lthe class of systems in NESSs which we will consider in this 
paper. We studied three classes. (A) Systems under a constant force, such as an electrically 
charged Brownian particle in a fluid subject to an external electric field £ [cf. Fig. [Ja)]; 
(B) Systems coupled to an oscillator, as, e.g. a Brownian particle confined by a harmonic 
oscillator, which is dragged through the fluid with a constant velocity v by an outside force [cf. 
Fig. [2ta)]; (C) Systems with two random noise sources. An example is two independent heat 
reservoirs at different temperatures, each containing a Brownian particle, which are coupled to 
each other harmonically, allowing an energy current from one reservoir to the other [cf. Fig. 
12 a)]. For each class we introduce not only Brownian particle models but also corresponding 
electric circuit models. In total we consider eight models: two of Class A, four of Class B and 
two of Class C. Each of these models can be described by a Langevin equation, which is given 
explicitly in a common form in the next section [3] by Eq. (flTl) . 

In the second part of the paper, in Sec. [3l we discuss the generalized Onsager-Machlup theory 
for a NESS and show how to obtain appropriate definitions of the heat and work from our 
general point of view. We first introduce the path integral method for the Langevin dynamics 
(flTl) . based on a Lagrangian to give a probability functional of paths. Then, following Onsager 
and Machlup, we write this Lagrangian for the NESS as a sum of two dissipation functions and 
an entropy production rate, where the latter allows us a definition of the heat in the NESS by 
integrating over time and multiplying by the temperature of a heat reservoir, connected to the 
systemQ By minimizing this Lagrangian we obtain the average path, which then leads to the 
non-negativity of the entropy production for the average path, i.e. the validity of the second 
law of thermodynamics for the average path. Finally, using the energy conservation law (the 
first law of thermodynamics), the work is obtained as a sum of the heat and the internal energy 
difference. This work consists of four parts: (i) work given by the partial time-derivative of 
the internal energy, (ii) work done by an external driving force, (iii) work caused by a time- 
irreversible force, and (iv) work by a temperature difference between reservoirs. 

In this part of the paper we also discuss in detail the role of time-reversal for our NESS Onsager- 
Machlup theory. We point out in detail the difficulties associated with the ambiguity of defining 
an appropriate backward path associated with a given forward path due to the presence of ex- 
ternal nonequilibrium parameters fi, e.g. a dragging velocity v or an electric field £, which 
specify the NESS forces or currents. To formulate this ambiguity, a time-reversal operator I± 

' Note that in this paper we consider the entropy production in NESSs, rather than a NESS entropy itself (|71. 



3 



is introduced which reverses (indicated by a hat) the direction of the (internal) motion (the ve- 
locity) of the system, as compared with that on the forward path, as well as a possible, but not 
necessary, reversal of the sign of the external nonequilibrium parameter (indicated by ± in 
i±)% Two possible definitions can therefore be given for the heat, corresponding to a + or — 
sign in I±, respectively, as well as for the work and the internal energy, leading to two possible 
expressions for the energy conservation law, or the first law as well as for the second law of 
(NESS) thermodynamics for each (not only the average) path. 

In Sec. IHwe discuss the nonequilibrium detailed balance relations and the transient fluctua- 
tion theorems HOl for work, which hold for both and All the above laws and relations 
are therefore unaffected by the ambiguities mentioned above, i.e. they are valid relations for 
the NESS, mdependent of the above ambiguities. For the transient fluctuation theorems this 
must be due to the fact that they are mathematical identities [[TT|. This means here that one 
obtains two formal identities, without the need to identify the appropriate thermodynamic work 
on physical grounds, as is necessary for a physical discussion of particular systems. This "uni- 
versal" validity of the transient fluctuation theorems could disappear for asymptotic fluctuation 
theorems lfT2ll for NESSs as was indeed shown in a previous paper for the case of a dragged 
Brownian particle model 

In the third part of the paper. Sec. [51 we will illustrate how the above mentioned ambiguities can 
be eliminated and lead to unique choices of the heat and work to maintain a NESS on a variety 
of models introduced in Sec. [21 Although these models are all linear we do not expect the nature 
of our considerations to be qualitatively changed, if non-linearities in the potentials, occurring 
in these models, are introduced. However, the dependence of, in particular, fluctuations on the 
properties of the stochastic noise is much less clear [[T3l[T4l| . 



2 NESS Models 

Before discussing our generalized Onsager-Machlup theory for NESSs, we introduce some typ- 
ical NESS models all described by Langevin equations. Using these models, we give concrete 
examples of external nonequilibrium parameters which specify the system in a NESS (so are 
zero at equilibrium) and change their signs with a reversal of the steady state force or current. 
These parameters play a crucial role in this paper and co-determine the choice of the proper 
time reversal procedure to calculate relevant work and heat to associate with a system, as will 
be discussed later. The internal energies for these models are also given in this section and will 
be used to determine the work to maintain a NESS in the following sections. As mentioned 
in Sec. [B we discuss these NESS models by separating them into three classes: Class A for 
models driven by a constant external force. Class B for systems coupled to an oscillator, and 
Class C for models with two random noises. 

- A time-reversal procedure involving a change of sign of a nonequilibrium parameter was already used before 
in shear flow systems |[8]|9l- 
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Figure 1: NESS models of Class A: (a) a charged particle driven by a constant electric field and 
(b) an electric circuit consisting of an inductor and resistor in series. For an explanation of the 
symbols, see the text. 



2.1 Class A: Systems under a Constant Force 



a) The first (and possibly simplest) example is an electrically charged Brownian particle in a 
fluid in a uniform electric field. The Langevin equation for this system is given by 

mXs = q£ — aXs + C (1) 

for the particle position Xs at the time s, where m is the mass, q the electric charge of the 
particle, 8 a constant external electric field, a the friction coefficient of the particle in the fluid, 
Xs = (Pxs/ds'^ and Xg = dxg/ds. Here, is a Gaussian-white random force whose first two 
auto-correlations are given by (Q) = and (CtiCt2) = (2a//3)5(ti — ^2), respectively, with _3 the 
inverse temperature of the heat reservoir and the notation (■ ■ ■) for an ensemble average|j The 
Brownian particle is driven by a constant force qS via the external field S which plays the role of 
the external nonequilibrium parameter in this model and vanishes at equilibrium. A schematic 
illustration for this system is given in Fig. [It a). The internal energy E of this system is given 
by 

E{xs) = -mil. (2) 

It is important to note that here we regards qS as an "external" driving force and its correspond- 
ing potential energy is not included in the "internal" energy E. In this system, the Brownian 
particle achieves a constant average velocity v = q£/a in a NESS. Note that a nonequilib- 
rium state driven by a constant force can be realized in variety of other ways, for example, in a 
Brownian particle under a constant gravitational force. 

b) As a second example we consider an electric circuit consisting of an inductor (with self- 
inductance L) and a resistor (with resistance R) in series IfTSl , as shown in Fig. [Ub). In this 

^ Note the coefficient 2a/ (3 in (CtiCt2) is due to the fluctuation dissipation theorem, which is, strictly speaking, 
justified around equiUbrium. In this report, we assume that it is stifl correct for our NESS models. 



5 



Model 




Class A 


Class B 


Brownian particle 




m 


a 


Cs 


qS 




V 


Electric circuit 


Qs 


L 


R 




V 


1/C 


I (parallel) 
CA (serial) 


Torsion pendulum 


9s 


I 


V 


Cs 




a 





Table 1: Correspondences of quantities in various NESS models. The external nonequilibrium 
parameters characterizing the deviations of the systems from an equilibrium state are 8 and V 
in Class A, and v, /, A and ^ in Class B, respectively. Explanation of symbols is in the text. 



circuit, the voltage V of the battery is equal to LIs+RIs+SVs with the electric current Ig through 
the resistor and a voltage random noise dVg in the resistor. Combining this with Ig = qs, where 
qs is the charge in the resistor, we obtain the Langevin equation 

Lqs = V - Rqs - SVs. (3) 

Here, we assume that 5Vs is a Gaussian-white random noise whose first two auto-correlations 
are given by = and {SVs5Vs') = {2R/ [3)5{s — s') by the Johnson-Nyquist theorem 

lfT6l [TTl . The external nonequilibrium parameter in this model is given by the voltage V of 
the battery. We note that the two Langevin equations O and ([3]) have the same form. We 
summarized correspondences of the quantities in these two equations in Table [T] Noting these 
correspondences, the energy E of this electric circuit model is given by Eq. Q with a replace- 
ment of m and Xg by L and qs, respectively, i.e. by E{qs) = (l/2)Lgf . 



2.2 Class B: Systems coupled to a Harmonic Oscillator 

As the second class of NESS models, we consider a system under an oscillating force. 

a) The first example in this class is a Brownian particle confined by a harmonic potential which 
is dragged by a constant velocity v in a fluid |l5l [6l [HI [191. The Langevin equation for this 
system is given by 

mxs = -K,{xs - vs) - axs + (s (4) 

for the particle position Xg with the oscillator spring constant k and the Gaussian-white random 
force Cs- In this model, the dragging velocity v plays the role of the external nonequilibrium 
parameter which is zero at equilibrium. A schematic illustration of this model is given in Fig. 
|2l^a). In this model the internal energy E of the particle is given by 

E(xs,Xs) = ^mxl + ^K{xg - vsY (5) 

where the second term on its right-hand side is the potential energy of the particle in the har- 
monic oscillator. 
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(a) (b) 




Figure 2: NESS models of Class B: (a) a particle dragged by a harmonic potential with a 
constant velocity, (b) an electric circuit with a serial inductor-resistor coupled to a capacitor in 
parallel, (c) a torsion pendulum confined by a spring with an external torque, and (d) an electric 
circuit with an inductor, resistor and capacitor in series. Symbols are defined in the test. 



Although the model described by Eq. dH) may be regarded as a Brownian particle model produc- 
ing a constant average velocity of the particle, like the electric field driven model described by 
Eq. ([U), we should notice there are clear differences between these two models. In this dragged 
Brownian particle model (Class B), the particle moves with a constant velocity even if there is 
no friction, since the average velocity of the particle is independent of the friction constant a. 
On the other hand, in the electric field driven model (Class A), the average velocity of the par- 
ticle will depend on the friction constant and if there is no friction then the particle accelerates 
indefinitely. We also note that there is no explicit time-dependent parameter in the force in the 
electric field driven model, while in a Brownian particle dragged by a constant velocity there 
is an explicit time-dependence in the force via vs in Eq. dH). These "simple" differences will 
manifest themselves in different definitions of work and heat in NESSs, as will be discussed in 
Sees. O and [521 

b) The same form as the Langevin equation dH) appears for an electric circuit in which an 
inductor and resistor in series are coupled with a capacitor in parallel [|20l , as shown in Fig. 
I2b). We first derive a Langevin equation for this system. Denoting the electric current through 
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the resistor by Ig = Qs with Qs the charge through the resistance, as shown in Fig. Ob), the 
voltage difference Vis between the ends of the inductor and resistor in series with the Johnson- 

Nyquist voltage fluctuation 6Vs as a Gaussian-white noise is given by Vis = LI s + Rh + ^Vs. 
On the other hand, the voltage V2s applied to the capacitor (with the capacitance C) is given 
by V2s = {Is — qs)/C where / is the constant electric current from the battery, as shown in 
Fig. I2b). Here, we used that the charge of the capacitor is given by Is — qg, i.e. the charge Is 
received from the battery minus the charge qs taken to the resistor. Using that V^is = V2s and 
Is = qs we obtain 

Lfjs = -^{qs - Is) ~ Rqs - SVs, (6) 

which is the Langevin equation for the charge qg. The external nonequilibrium parameter of 
this system is given by /. Note that, different from the previous electric circuit model in Class 
A [cf. Eq. ([3])] in which the voltage of the battery is constant, in this electric circuit model, 
described by Eq. Q, the electric current / from the battery is assumed to be constant. The 
energy of this system is given by Eq. © with a replacement of and V by L, qg, 1/C 

and /, respectively (cf. Table [I])- 

c) The third example of Class B is a torsion pendulum under an external torque in a fluid [!2T]| . 
A schematic illustration of this model is given in Fig. ^c) as a rod with the total moment of 
inertia X, rotating around its center with a spring functioning as a torsion. The time-derivative 
XOg of the angular momentum T9s for an angular displacement 9g must be equal to the torque 
applied to the rod, so that the equation of motion for 9g is given by the Langevin equation 

X9g = -a9g + Mg- v9g + C (7) 

where v is the viscous damping, a the elastic torsional stiffness of the pendulum, and M.g the 
external torque. For this model, we consider the case of a linear external torque of 

Mg = ^s (8) 

with a force constant ^. Since the pendulum is driven externally by the torque ([8]), its coefficient 
^ plays the role of the external nonequilibrium parameter. In this model the internal energy E 
of the particle is given by 

Ei9g,9g) = ^I9'^ + ^a9l (9) 
as the sum of the kinetic energy and the torsion energy. 

It is important to note that although the Lagrangian equation (|7]) with the torque ([8]) has the same 
form as Eq. (H]) (cf. Table [I]), the energy Q in this driven torsion pendulum model does not 
have the same form as the energy ([5]) for the dragged Brownian particle model. As shown later 
in this paper, this difference of energy also appears as a difference in the definition of the work. 
Another difference between these two models is that for the driven torsion pendulum model 
the average internal energy increases with time in a NESS, leading to a time-proportional work 
rate as will be discussed in Sec. 15. 2[ while in the dragged Brownian particle model the average 
internal energy is independent of time in a NESS with a constant average work rate. 
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Figure 3: NESS models of Class C: (a) two harmonically coupled particles connected to two 
heat reservoirs with different temperatures, and (b) an electric circuit with two resistors. 



d) As the last example in Class B, we consider an electric circuit consisting of an inductor, 
resistor and capacitor in series with a time-dependent applied voltage Vg = As with a constant 
A Il20l . [See Fig. [2td) for a schematic illustration of this model.] The Langevin equation for 
the charge Qs is given by 

Lqs = + As - Rq, - 6Vs. (10) 

In this model, the equilibrium state is realized when A = 0, so that A is the external nonequi- 
librium parameter. Note that the Langevin equation (flOl) has the same form as Eqs. (H)), ^ and 
(|7]) (cf. Table [I])- The energy of this system is given by Eq. ^ with the replacements of m, Xg 
and K by L, Qs and l/C, respectively (cf. Table[I]). 



2.3 Class C: Systems with Two Random Noises 

As the last category of NESS models discussed in this paper, we introduce stochastic models 
with two random noises. 

a) The first example in this category consists of two Brownian particles coupled by a spring, 
where each particle is confined to a heat reservoir at a different temperature (cf. Refs. |fT9l[22l ). 
We give a schematic illustration of this model in Fig. Ha). The Langevin equation for the 
positions xu and X2s of the first and second particle, respectively, is given by 

mXjs = -K {Xjs - Xks) - aijs + Cjs (11) 

with j ^ k, j = 1,2 and k = 1,2. Here, and (2s are two independent Gaussian- white random 
forces at different temperatures T + AT/2 and T — AT/2, respectively, so that (Qs) = and 
iCjsCj's') = i2a/pj)5jf5{s-s') with the inverse temperatures /3j = {A;s[T+(-l)J+^AT/2]}-^ 
For simplicity we assumed identical masses and friction coefficients for the two particles. In 
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this system, a NESS is sustained with an energy transfer between the two reservoirs due to the 
temperature difference AT as an external nonequilibrium parameter. The internal energy E of 
this system is given by 

E{ks, Xs) = ^mx'^js + {^is - X2sf (12) 
i=i 

Withx^ = {xis,X2s)- 

b) We can consider a similar stochastic system with two random noises in an electric circuit as in 
Fig. Sb), which is like that in Fig. Ob), except for an additional resistance next to the battery. 
We denote by lu = Qu the electric current associated with charge qu through the resistor 1 
(with resistance -Ri) next to the coil and by = q2s the electric current associated with charge 
q2s through the resistor 2 (with resistance R2) next to the battery. The charge on the capacitor is 
given by q2s — qu, so that the voltage drop in the capacitor is {q2s — qis)/C. This voltage drop 
is equal to the voltage difference Liu + Rihs + SVu between the ends of the inductor and the 
resistor 1 in series, and also to the one V — R2I2S — SV2S between the ends of the battery and the 
resistor 2. Here, SVu and 5V2s are independent Gaussian- white random noises in the resistors 
1 and 2, respectively, with {6Vjs) = and {6Vjs6Vj's') = {2Rj/ f3)6jj'6{s — s'), j = 1,2 and 
j' = 1,2. Using these voltages, the Langevin equation for the charges qjs is given by 

Lqis + Riqis + SV^, = V - /^afe - ^^^2. = (13) 

In this system the electric current is driven by the voltage V , which is the external nonequilib- 
rium parameter. The energy E of this system is given by 

I-./- \ 1 r -2 , i.'ils — <i2sf 

E{qs, qs) = ^Lq^ + ^ (14) 

with = {qis,q2s)- 



Although we categorize the above two models as a single class C, the corresponding Langevin 
equations ([TT]) and (fT3l) do not have exactly the same form, different from the models in Class 
A or B. However, we can introduce a single Langevin equation, which reduces to Eqs. (fTTI) and 
(fT3l) as special cases: 

rUjXjs = T5j2 - K{Xjs - Xks) - CtjXjs + Cjs, (15) 

with j ^ k, j = 1,2 and k = 1,2, a constant T and two independent Gaussian- white random 
noises and ^2^ at different temperatures T + AT/2 and T — AT/2, respectively, so that 
{(j,) = and ICjsCj's') = {2a,j/ (3j)5jj'5{s - s'). We take ^l = {AT,T) with two external 
nonequilibrium parameters AT and T in this model. The internal energy E of this system is 
given by 



E{xg, Xg 



^ 1 



^rrijXjg + -K (xi, - X2s 



(16) 
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Eqs. (fTSi) and (fT6l) become Eqs. (fTT)) and (fT2)) . respectively, in the case of mi = m2 = m, 
tti = 0^2 = a and F = 0, while they reduce Eq. (fT3l ) and (fT4l) . respectively, in the case of 

Xjs = Qjs, mi = L, m2 = 0,T = V Oj = Rj, k = 1/C, Qs = -SVjs and jSi = (32 = (3 (i.e. 
AT = 0). 

After having introduced here the NESS models which we will use in this paper, we now discuss 
the theory which we will apply to them. 



3 NESS Onsager-Machlup Theory 

In this section we discuss a generalized Onsager-Machlup theory for NESSs, or simply the 
NESS Onsager-Machlup theory, for linear stochastic models, including those introduced in the 
previous section. 



3.1 Path Integral Approach to Stochastic Dynamics 

We can write the Langevin equations for all the models in the previous section [2l in the form: 

j = 1, 2, ■ ■ ■ , A^, for a system with N degrees of freedom described by = (xis, X2s, ■ ■ ■ xns)- 
Here, Xjs is a position (charge), nij a mass (self-inductance), aj a friction coefficient (resis- 
tance), and Fj a mechanical force (voltage) including the external nonequilibrium parameter rj 
in Brownian (electric circuit) models, respectively. Furthermore, Qs incorporates a Gaussian- 
white random noise, so that {Qs) = and {CjsCj's') = (2aj//5j)5jj'5(s — s') with the inverse 
temperatures Pj. 

Note that systems with N degrees of freedom have already been considered in Onsager and 
Machlup's original theory using N independent variables ai, a2, ■ ■ ■, for any integer number 
N BH. (Also see the introduction of Ref. jSl.) However, we emphasize that for our models 
introduced in Sec. [2] it is enough to consider = 1 or 2, i.e. = 1 for Classes A and B or 
= 2 for Class C, although our general theory developed in Sees. [3] and |4] is formally correct 
for any A^. 

Although the mechanical force Fj(xs, s; ij) could be a nonlinear function of in the Langevin 
equation (flTl) . we restrict ourselves in this paper to functions linear in Xjs, consistent with the 
linear Langevin equations used in the previous section [2l which is sufficient for the purposes of 
this paper. Thus we will impose the condition 

(F,(x„s;77)) = F,((x,),s;r?) (18) 
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for the ensemble average of the force -Fj(xs, s; t]) which is linear with respect to x^. We will 
use this condition (fTSl) to discuss the second law of thermodynamics in Sec. |3.3| of this paper. 
Otherwise, the linearity of the force Fj is not used in the general theory developed in this paper. 

Similarly, we assume for the models of Class C that the temperature difference between the two 
heat reservoirs is small, so that 



AT, 



T 



« 1 



where T is the average temperature T = (l/N) J2f=i ^^'^ the deviation ATj =Tj — T 

of the temperature Tj = l/{kBPj) of the j-th reservoir from T with the Boltzmann constant 
ks- We will calculate quantities like work and heat up to the lowest non-vanishing order in 
\ATj/T\. Here, AT = (ATi, AT2, • ■ ■ , AT^) plays a role of the thermal nonequilibrium pa- 
rameter and combining it with the mechanical nonequilibrium parameter 77 we obtain the total 
external nonequilibrium parameter /x = (77, AT). 



(19) 



For later use, we now implement the stochastic dynamics, given by the Langevin equation 
(flTI) . using a path integral approach. Thereto, we note that the probability functional Pc({Cs}) 
of the Gaussian-white random noise Cs = (C1S7C2S, -Xms) is given by ^^({Cs}) = exp{ 
— Yl!j=i[Pil{^^j)] It '^^ (js} "^ith the normalization constant C^. By inserting (^^ from the 
Langevin equation into this functional 'Pf({Cs}) and interpreting V(^{{Cs}) then as the 
probability functional Vx{{^s}) for the path {:x.s}se[u,tf], we obtain [|6l, apart from a normal- 
ization constant. 



/ £(xs,X5,Xs,s;/x) 



(20) 



where the function £(xs, x^, x^, s; n) of x^, x^, x^j and s is a Lagrangian given by [[6l 



■^(Xs, Xg, Xg, S, /J.) 



TV 



rrii 



X 



— Fj(xg,s;j7) H X 



a 



a,- 



(21) 



with the normalization constant C^. [See also, for example, Ref. Il23l for a derivation of the 
probability functional (|20)) via the Fokker-Planck equation corresponding to the Langevin equa- 
tion (fT7]) .1 



3.2 Time-Reversal in NESS 



Time-reversal plays a crucial role in nonequilibrium thermodynamics. For example, the Onsager- 
Casimir symmetry relations between two linear transport coefficients have a different sign, de- 
pending on the behavior of thermodynamic variables under time reversal Il24ll25l . Moreover, 
in the Onsager-Machlup fluctuation theory around equilibrium [)4J, the entropy production rate 
is directly related to the difference of a Lagrangian for a forward path and the corresponding 
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Lagrangian for a time-reversed (or backward) path. In the next subsection we will discuss a 
generalization of this argument for the entropy production around equilibrium states to NESSs. 
But first, before such a discussion, we must clarify an essential difference of a time-reversal 
procedure in NESSs from that in equilibrium states. 

In equilibrium states, the time-reversal of the dynamics is unique and is simply given by a 
change of sign of the particle velocity x^. On the other hand, the time-reversal in NESSs is not 
unique. This is due to the two independent kinds of motions in such states: an internal intrinsic 
particle motion given by x^, but, in addition, by an externally induced motion characterized 
by the external nonequilibrium parameter /x. Therefore, in NESSs, we have two choices for a 
time-reversal procedure of the dynamics: either a change of sign of x^ only, which has always 
to be done to obtain a time-reversed path, or a change of the the signs of both x^ and /x. To 
discuss these two time-reversal procedures explicitly, we introduce the time-reversal operator 
J± for NESSs by 

/±X({xJ; /.) = X({xt+t„_.}; ±/i) (22) 

for any functional X({xs}; /i) of the path {xs}sg[t(, t] and the external nonequilibrium parameter 
Under this time-reversal operation, the direction of motion of the particle on the forward 
path {xs}sg[t„,t] in the functional X({xs}; /x) is transformed into {yit+to-s] s£[to,t] with the same 
geometry of the path but with the initial and final positions on the forward path (on the time- 
reversed path) given by Xf^, and x^ (xt and xt„), respectively. This time-reversal operation 
for the internal motion represented by the particle position x^ is indicated by the hat " on the 
operator I±. On the other hand, the other time-reversal procedure associated with the external 
nonequilibrium parameter /i as well is referred to by adding the subscripts ± in the operator I±, 
so that /_ (/+) does change (does not change) the sign of the nonequilibrium parameter under 
this time-reversal operation. 

So far, we have chosen the initial time to and the final time t independently, to make clear their 
roles. However, for convenience, in the remaining part of this paper, we choose, without loss of 
generality, the origin of the time in the middle of the initial time to and the final time t, so that 
to = — t. By taking this origin of the time, the length of the time interval for s G [to, t] is given 
by t - to = 2t. 

We now discuss some properties of the time-reversal operator I± useful for later. First, by Eq. 
(I22l) the time-reversal operator I± satisfies the relation 

4' = 1. (23) 
Second, it can also be shown for this time-reversal operator that 

/± j dsY{xs,ics,^s,s;fi) = j ds ^{ ^^^' ^ ^^^^ 



ds F(x^, -Xs, Xs, -s] ±/x) , (25) 



Here and in the rest of the paper we adopt the convention that any equation containing the symbols ± on the 
left- and right-hand sides, denote two equations, one with only the upper symbol (+ in ±) and the other with only 
the lower symbol (— in ±). 
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for any function F(xs, x^, x^, s; fx) of x^, x^, x^, s and fi. Eq. (l25l) means that the effect of the 
time-reversal operator i± on a functional of the form J^^ F(xs, x^, x^, s; /x) is expressed not 
only by the change x^ — — x^ of the (internal) particle velocity, but also by the changes s —s 
of the explicit time-dependence, and by /x — ±/x for the external nonequilibrium parameter in 
the function Y^Xg, Xs, x^, s; /x). 



3.3 Dissipation Functions, Entropy Production and the Second Law of 
Thermodynamics 



In this and the next subsections, using the time-reversal procedure introduced in the previous 
subsection 13 .21 we formulate a generalized Onsager-Machlup theory for NESSs in three steps: 
[i] calculation of the entropy production rate as the time-irreversible part of the Lagrangian, 
leading to the second law of thermodynamics (Sec. 13.31) . [ii] introduction of the heat via the 
entropy production rate (Sec. 13.41) . and [iii] introduction of the work from the heat and an 
internal energy difference using the energy conservation or the first law of thermodynamics 
(Sec. m. 

To discuss the first step for the NESS Onsager-Machlup theory, we separate the Lagrangian C 
into a time-reversal invariant (even) part and a time-irreversible (odd) part as 

1 



•^(Xs5 x^, x^, s, /x) 



2k 



B L 



$±(x^, x^, s; /x) - S'±(x^, x^, X,, s; /x) 



(26) 



where ^± and S± are defined by 

<l'±(xs,Xs,x^, s; /x) = -/cs [/:(x^,x^,x^,s;/x) + /:(xs,-Xs,Xs,-s;±/x)] , (27) 
S'±(xs,x„X5,s; /x) = /i;B[/^(xs,Xs,x^,s;/x) -/:(x^,-Xs,Xs,-s;±/x)]. (28) 

From Eqs. (l27l) and (|28|) . using the property (l25l) for the time-reversal operator I±, we obtain 



/± / ds $±(x^,x^,x^,s;/x) 



/± / ds S'±(x^, x^, x^, s; /x) 



ds $±(x,,x,,x„s;/x), 



ds S'±(x^, x^, x^, s; /x) 



(29) 
(30) 



for the time-reversal part $± and the time-irreversible part S± of the Lagrangian £, respectively. 

A major point in the Onsager-Machlup theory is then that the odd part of the Lagrangian is 
identified with the entropy production rate flU. To discuss the physical interpretation for the 
odd part S± of the Lagrangian C for NESSs, we first have to give more explicit forms for S± 
and $±. By inserting the Lagrangian (|2TI) into Eqs. (|27l) and (l28l) . we obtain 



$±(xs,Xs, s; /x) 
*S'±(x5, x^, Xg, s; /x) 



<l>i^^(is,x„s;/x) + $^^^(i„x„s;/x) + <l>^^^(i„ x„ s; /x), (31) 

N 



.(2)/ 



.(3)/ 



=1 ^ 



X 



a,- 



+$i^^(x^,x^,s;/x) 



(32) 
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where \ k = 1, 2, 3 are defined by 



TV 



E 

TV 



2T, 



--F£)(x„.;^) 



n 2 



2ajTj 



(33) 
(34) 



-, a,- AT,- 



t 2 T - AT/ 



a; 



H Tj(x^, -s; ±77) Xj 



a 



a,- 



and F^^ and Tj^-* are defined by 



[T,-(x^,s; ?7) 



2 

- [T,-(x„ s; rj) - T,-(x„ -s; ±r])] 



(35) 

(36) 
(37) 



as the even (e) part and the odd (o) part of the force Fj = Fj2 + Fj'^, respectively, under 
the time-reversal procedures s —s and either /x +/x or /i ^ — /x. Here, and ^^2^ 
correspond to the dissipation functions in the Onsager-Machlup theory [5] and $^ = 0. Using 
Eqs. (|26|) and (|3TI) the Lagrangian can be represented as the sum of the dissipation functions, 
and the minus entropy production rate, i.e. C = -[1/(2/cb)] + + <l>^^ - S±] for 
NESSs, in a similar way as in the Onsager-Machlup theory for equilibrium states. 

Eqs. (1331) and (|34|) show that the dissipation functions and are always non-negative 
and time-reversal invariant, i.e. ^^^^ > and /± /^^ (is ^li^'^ = /^^ ds k = 1,2. This non- 
negativity of the dissipation functions is directly related to the non-negativity of the average 
entropy production rate, namely the second law of thermodynamics, in the linear regime. To 
show this, we note that in our NESS Onsager-Machlup theory, the average path {(x5)}se[_f,j] is 
given by the variational principle 



£(xs5 ^s, s] //) = minimum for x^ = (x^), 



(38) 



leading to the average Langevin equation m{xjs) = -Fj((xs), s; rj) — aj{xjs) using Eqs. (fTSl ) 
and dH]), i.e. 



(ijs) - -i^?((x,), s; v) = - k?((x.), s; v) - (39) 

with Eqs. (l36l) and (l37l) . Using Eq. (|39| ), we see that 2 times the dissipation functions (l33l) and 
(|34|) and the entropy production rate (l32l) minus coincide with each other for the average 
path, i.e.: 



S'±((x^), (xs), (x,), s; /i) - <l>^^((x^), (xs), s; /x) 

= 2$£)((x,), (x,), s; /x) = 2$f ((x,), (x,), s- /x) 



(40) 
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Combining Eq. (l40l) with the non-negativity of the dissipation functions and we obtain 

^±((ic,), (i,), (x,), s; ,x) - <^^l\{ks), (x,), s; m) > 0. (41) 

This is a statement of the second law of thermodynamics in our generalized Onsager-Machlup 
theory for NESSsjf] We note that Eq. (|4T1) expresses a non-negativity of S± — for the 
average path {(xs)}sg[_t,t], but the quantity S± — itself can be negative for other paths, in 
contrast to the dissipation functions and ^^f^ , which are always non-negative for any path. 



3.4 Heat, Energy, Work and the First Law of Thermodynamics 



We now discuss, as the main results of this paper, the appropriate heat and work to maintain the 
NESSs from the entropy production rate S± discussed in the previous subsection |3.3[ 

Using the entropy production rate S±, we introduce the heat Q± produced in the system on the 
trajectory {x,},g[to,j] by 



Q±({x4;/x) = T ds S±{xs,±s,Xs,s;n). 



(42) 



-t 



InsertingEq. ([321) into gll) and using Tr^ = T [l-ATj/T] + 0{\ATj/T\'^) and the condition 
(fT9l) we obtain 



Q±({xs};m 

N 



F^2{^s,s;r]) -rrijXj 



rv ■ -J 



a 



J. rrij 
iCjs H Fji^s, -s; ±r?) 



1 2 



X 



a 



+0 



AT, 



T 



(43) 



as a concrete form of the heat Q± up to the first order in ATj/T. The term involving AT, on the 
right-hand side of Eq. (l43l) gives the heat produced by the system with temperature differences 
between reservoirs. 



We now discuss properties of the heat Q± from its definition (|42)) . We first note that due to Eqs. 
(l30l) and (|42)) the heat Q± is anti- symmetric under time-reversal, i.e. 

/±Q±({x.}; /x) = -Q±({xJ; /i). (44) 

^As will be shown in Sec. |5] the term ^f^' disappears for the physical entropy production rate to maintain a 
NESS for all models introduced in Sec. |2] 
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Using Eqs. (1201) . (1281) and (|42|) we can also show that 

/t 
ds [/:(xs,Xs,Xs,s;/^) - £(xs,-Xs,Xs,-s;±/x)] (45) 

= r^ln . (46) 

/±Pa;({x,};/x) 

with /5 = l/{kBT). Eq. (1451 ) connects directly the heat Q± with a time-irreversible part of 
the Lagrangian C. Eq. (|46l) implies that the behavior of the dynamics under time-reversal 
makes the probability functional Vxi{xs}; /i) for the forward path not equal to the correspond- 
ing probability functional I±Vx{{^s}', /^) for the backward path and a nonzero heat is due to 
this non-equality. 

We now proceed to introduce the internal energy and then the work using the energy conserva- 
tion law as a relation among the heat, the internal energy and the work. To introduce the internal 
energy, we first separate the even part Fj^ of the force Fj into a force due to the internal po- 
tential U± and a force fj± due to the external driving force (e.g. an external electric force on a 
charged particle): 

F;^'(x„ s; rj) = + /j±(x„ s; r?). (47) 

CXg 

The separation of the force into a potential force and an external driving force, like in Eq. (l47l) . 
has already been used before, cf. Ref. ||26l . Using the potential U±, we next introduce the 
internal energy E± by 

^ 1 

E±(xs,x,,s;77) = ^-mj\xjf + U±{:s.s,S]'n) (48) 

i=i 

as the sum of the kinetic energy J2f=ii'^/'^)''^j\^js\'^ '•^^ potential energy U±. Using this 
energy, we introduce the energy difference AE± by 

AE± = E±{±t, xt, t- rj) - E±(x_i, x_(, -t; rj) (49) 

as the difference of the internal energy at the final time t and the initial time to = —t. 

Now we will introduce the work. We first note that in physical processes the external work 
is transformed into heat and a change of the internal energy, as expressed in the energy con- 
servation law. From this, the work W± done along the trajectory {xs}se[-tt] is then defined 
by 

W±({x,}; m) = Q±({xJ; fj,) + AE±. (50) 

Eq. (l50l) leads to an expression of the first law of thermodynamics for NESSs, by taking its 
functional average. 

As a remark about the internal energy and the work, in the above argument we first introduced 
the heat Q± by Eq. (|42l) . then separated it into the energy difference AE± and the work W± 
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via Q± = W± — AE±, i.e. Eq. (l50l) . However, this separation of the heat into the work and 
the energy difference is not unique, due to the non-uniqueness of the separation (|47l) of the 
force Fj2 into an external force fj^ and a potential force —dU±/d^s, which introduces the 
potential U±. This non-uniqueness of the potential actually happens in the models discussed in 
Sec. 12.21 where the dragged Brownian particle model and the driven torsion pendulum model 
are described by the same Langevin equations, but have different internal energies. This non- 
uniqueness, or ambiguity, in the introduction of a potential, by a separation of the force on 
the particles into a force due to an internal potential and an external driving force, can only 
be resolved for specific models, it seems, on physical grounds, rather than by mathematical 
arguments alone. 

A related remark about the above argument to introduce the work and the energy in a NESS 
is that we assumed that there is no contribution from the odd part F^'^ of the force Fj to the 
internal potential U±, therefore to the energy E±. This implies that the internal energy E± must 
be time-reversal invariant, i.e. 

^±(-X3,X3, -s; ±77) = E±(xs,x^,s; J7). (51) 

Using Eq. (|5T1) we obtain 

J-t ds 

i.e. the energy difference AE± is anti-symmetric under time-reversal. Eqs. (|44)) . (l50l) and (|52l) 
lead to 

4W±({xJ; /.) = -W±({xJ; m), (53) 

so that the work done on a backward path has the same magnitude but the opposite sign of the 
work done on the corresponding forward path. Eq. (l53l) will play an important role to derive 
the correct work fluctuation theorem in Sec. 14. 2[ 



To discuss the physical meaning of the work W± defined formally by Eq. (|50l) . we now give it 
in more explicit form. Inserting Eqs. (|43] ) and (|49l ) into Eq. (|50l) we obtain 



W±({xJ;a.) = Wf)({xJ;^) + Wy^({x,};^) + Wr({x,};r,) 

2^ 



;(/), 



+Wi*H{xJ;/x) + 



AT, 



T 



(54) 



where wi'\ >vi°^ and wi*' are defined by 



* dE±{±s,Xs,s;r]) 

■t 



ds 



W^J\M;ri) = y^l dsf,±{^,,s;rj 
,=1 



(55) 
(56) 
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Wi°^({x.};r/) 



N 



1 



ds 



7'^^ J CO ■J ^ 



■5; J?), 



(57) 



.3=1 



ds 



X 



s; 77) 



+ + 



±1 



J 2 T 



(is 



a 



-F,{^s 



-s; ±77) - 



m 



(58) 



[See AppendixlAlfor a derivation of Eq. (|54|).1 In Eq. (|54|) the total work W± has been separated 

The first part W± comes from the partial time- 



>vi^\>vi°^ andWf . 



This is the work used in Refs. ||27ll28l . The second part W^f^ 



into the four parts: W± 
derivative of the energy (e) E± 

is the work done by the external driving force (/), f±, while the third part W± ^ of the work is 
due to the odd (o) part Fj'2 of the force. We remark that this third part of the work includes 
a d'Alembert type force — mx^ as noted by Onsager and Machlup [4] and also by the authors 
jSllll. The last part yV± is due to the temperature (t) differences AT, among reservoirs. In 
Sec. [51 we will consider concrete examples for the four different kinds of works 'W^\ W^'^^ 



± 



W± and yV± using the NESS models discussed in Sec. [21 From the explicit form ([54l) of the 
work, together with Eqs. ([55l) . ([56l) . ([57l) and ([58l) . we can show that the work W± is a purely 
nonequilibrium quantity and vanishes at equilibrium, W± = 0, because the energy E± does not 



have an explicit time-dependence (so that W. 



(so that Wi"^ 



0), /j± = (so that Wi^^ = 0), = 

0) and ATj = (so that =0). At equilibrium, where Onsager and 
Machlup formulated their fluctuation theory [[H, the energy balance equation is simply given by 
Q± = -AE± from Eq. ^ using W± = 0. 



Finally, we want to make some remarks about the ambiguities caused by the non-uniqueness 
of the time-reversal operator I± in NESSs. In this section we have discussed how to introduce 
thermodynamic quantities like the entropy production rate, the heat, the internal energy and the 
work, etc., suitable for a NESS. However, so far we could only specify for each of them two 
possibilities, e.g. 5+ or for the entropy production rate, W+ or W_ for the work, and so 
on, due to the two possibilities (+ or — ) contained in the time-reversal operator Ij^ for NESSs. 
The above arguments do not answer the question, which of these two actually represents the 
physical work, etc. to maintain a NESS. Both works W+ or W- and heats or Q_ satisfy 
the energy conservation (first) law ([50l) and both entropy production rates S+ and satisfy the 
second law of thermodynamics ([4TI) . One of the possible criteria to choose the physical work, 
etc. to maintain a NESS, is that we choose one of the two time-reversal operators /+ and /_ in 
such a way that the internal energy E must have the time-reversal symmetry ([STl) . In this way, 
for example, as will be discussed in Sec. 15.21 we can choose the correct time-reversal operator 
/_ for the dragged Brownian particle model in the laboratory frame, noting that the energy © 
has the time-reversal symmetry for the time-reversal operator /_ with jj, = v. Therefore, the 
quantities with the suffix like W_, and Q_, give the physical quantities to maintain 
a NESS for the dragged Brownian particle model. Another criterion to choose the physical 
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quantities to maintain a NESS is that the average of the work and the heat must be strictly 
positive, because in the NESS we always need to do positive average work (as well as remove 
positive average heat) to sustain the system in a NESS. This condition also leads to a strictly 
positive entropy production in NESSs. As we will see in SeclSl for some NESS models, only 
one of the averages of S+ and S- is strictly positive while the other vanishes in NESSs. In such 
a case, we can regard the strictly positive entropy production as the physical one to maintain a 
NESS and we can choose the heat, energy and work, etc., corresponding to this physical entropy 
production. In Sec. [51 we will illustrate the resolution of the above ambiguities in the choice 
for the physical heat and work, etc, using the systems discussed in Sec. [21 To discuss this point, 
hereafter we will use the terminology of the "physical NESS" heat and work for the heat and 
work to sustain a NESS, respectively. 



4 Fluctuation Theorems 



So far, we have discussed the definition of the entropy production, the heat, the internal energy 
and the work in NESSs. These quantities are introduced as functions of a path, so that they 
include information not only of their ensemble averages (over all paths) but also of their fluc- 
tuations. In this section we discuss their fluctuating properties in terms of fluctuation theorems 
using our NESS Onsager-Machlup theory. We restrict our arguments to the NESS detailed bal- 
ance condition and the corresponding transient fluctuation theorem where the initial state is an 
equilibrium state. For other fluctuation theorems, like the asymptotic fluctuation theorem for 
any initial state, we refer to our previous papers [[51 [H for a dragged Brownian particle model. 



4.1 Nonequilibrium Detailed Balance Relation 

Up until now we have emphasized the role of the work W± to distinguish NESSs from equi- 
librium states. However, this work also plays an important role in a detailed balance condition 
for NESSs, which we call the nonequilibrium detailed balance relation. This relation has been 
already discussed in Ref. [3] for the dragged Brownian particle model and was used there to 
derive transient fluctuation theorems for work. In this section we generalize this to the classes 
of NESS models discussed in Sec. [2l 

The nonequilibrium detailed balance relation including the work >V± can be derived from Eqs. 
((461), and ^ as 

= Q±{y.t, xt, t; rj) /±'P^({x,}; ^l) (59) 
where Q±{'k.s, x^, s; rj) is a canonical-like distribution function defined by 

^±(x^,x^,s; J7) = exp {/9 [J^±(s; t?) - E±(x,, x,, s; t?)]} (60) 
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with T± given by 



(61) 



to normalize the canonical-like distribution function Q±{'Xs, x^, s; rj). Here, in Eq. (|59| ). AJ^± is 
defined by 

AjC-^ = r,) - 77), (62) 

i.e. the difference of J-'± between the initial time to = —t and the final time t. 

We will now show that the nonequilibrium detailed balance relation (|59l ) reduces to the well- 
known equilibrium detailed balance condition. We first note that at equilibrium AjFj. = 0, 
yV± = and that there is then also no distinction between the two time-reversal operators, i.e. 
1+ = I^, since the external nonequilibrium parameter is zero: = 0. Thus, at equilibrium we 
can write g± = /'"^'''(x^, x^) using an equilibrium canonical distribution function /^^''^{ics, x^). 



Secondly, we introduce the transition probability P 



from an initial (i) point 



(xj, Xj) at time to to a final (f) point {-Kf,i<.f) at time t, which is given in terms of the proba- 
bility functional ^^({xj) for paths {xj^g^^o^t] by P 



Xj , Xj 

to 



p{xt,xt)={^f,yif) 



I?x, 



Px({xs}), i.e. taking the path integral (denoted by "Dx^) of Vx{{^s}) over all paths {xs}sg[(,)_t] 
under the conditions (xtg,X(„) = (xj,Xi) and (xt,xt) = (x/,x/). Using this and inserting 
the above equations AJ^± = 0, W± = and g± = /["^^l for equilibrium states into Eq. 

/[-^l(i_i,x_i) 



591 ). we obtain the equilibrium detailed balance condition P 



X*, X( 



X_t, X„( 

-t 



/[^^l(Xi,Xi)P 



Xt, X( 

-t 



4.2 Transient Fluctuation Theorems for Work 



The nonequilibrium detailed balance relation (1591 ) imposes a special relation on the probability 
distribution function for work, which is called the transient fluctuation theorems llTOl . To discuss 
this theorem, we introduce the probability distribution function Pu,±{W, t; /x) for the work >V± 
as 

Pu.±{W, t-f,) = { 6{W - >V±({x,}; . (63) 

Here, (•••)( indicates the functional average, which is defined by 

r r /•(xt,xi)=(xj,x^) p p 

{ X({xJ) ), = d^f / d±f / Dx, / dxi / d±i 

J J J(x_t,X_i) = (Xi,Xi) J J 

xX({xJ)P,.({xJ)/(i„ x„ -t) (64) 
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for any functional X({xs}) of the path {xs}se[-t,t], where /(x.j, Xj, —t) is the initial distribution 
function of the initial position Xj and the initial velocity Xj. The work distribution function (|63]) 
can be rewritten in the form (cf. Ref. ||5l ) 

1 r+oo 

Pv.±{W, t;ti) = — da e''''^S^±{ta, t; fi) (65) 
with £±{a, t; fi) defined by 

£:^±(a,t;M) = (e-'^^±«->^/^)»^. (66) 

The function £u,±(ia, t; /x) can be regarded as a Fourier transformation of the work distribution 
function P^±{W, t; fi), as well as a generation function for the work W±({xs}; /i). The work 
distribution function Pu,±iW, t; /x) was obtained explicitly for all W and t by carrying out path 
integrals for dragged Brownian particle models as in Refs. [I5l[6l. 



In the remaining of this section, we assume that the initial distribution function /(x„t, x„t, —t) 
at the initial time to = —t is given by the canonical-like distribution function (|60l) . i.e. 

/(x_t, x_t, -t) = g±{±^t, x_j, -t; rj). (67) 

In that case, we obtain 

£^±((3 - a, t- /i) = e-^^^±4^«,±(t^, fi). (68) 
Eq. (1681 ) leads then to two generalized transient fluctuation theorems: 

for the work distribution function Pw±(W, t; /x). [See Appendix |B] for a derivation of Eqs. (l68l) 
and (l69l).1 We emphasize that Eq. (l69l) is a relation for work fluctuations described by the 
distribution function Puj±{W, t; n), for the special initial condition (l67l) [cf. Eq. (|60l) 1. 

The two generalized transient fluctuation theorems (|69| ) reduce both to the usual transient fluc- 
tuation theorem ifTOl when the energy E± is the equilibrium energy independent of the external 
nonequilibrium parameter /x [so that then the initial distribution function (|67l ) is an equilibrium 
canonical distribution function f^^'^^ (x^, x^)] and also the condition AjF± = is satisfied]^ Note 
that Eq. (|69l ) implies two transient fluctuation theorems; one for P^^iW.t] n) and the other 
for Pw_{W, t; fi), as mathematical identities satisfied at any time for a canonical-like initial dis- 
tribution. Both these transient fluctuation theorems could be checked experimentally. These 
two different transient fluctuation theorems, for example, for the physical NESS work and for 
a work to overcome the friction in the dragged Brownian particle model, have already been 
discussed in Ref. Q. 

^ The energy E is independent of the external nonequihbrium parameter fx for all the models discussed in 
Sec. |2]of this paper, except for the dragged Brownian particle model in the comoving frame and its corresponding 
electric circuit model, when g±(xs, Xs, s; Ty) is an equilibrium canonical distribution function. In addition, the 
condition AJ^± = is satisfied for all the models discussed in this paper. 
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Finally, one may notice that the transient fluctuation theorems (l69l) have a similar form as the 
so-called Crooks theorem ||29l . which is a type of transient fluctuation theorem involving a free 
energy difference. However, Eq. (|69l) is not exactly the same as the Crooks theorem. First, in 
the Crooks theorem the work is always given by a time-integral of the partial time-derivative of 
the energy, like in Eq. (l55l) . while in Eq. ( l69l) the work W± does not have such a simple form, 
noting that the work (|54l) can contain, in general, other contributions, as given by Eqs. (|56l) . 
(|57] ) and (|58l ). Second, the energy E± is, in general, not the equilibrium energy because it can 
include the external nonequilibrium parameter fi, so that the quantity J-'± defined by Eq. (|6T| ) 
does not have to be an equilibrium free energy as appearing in the Crooks theorem. 

5 Illustration of the Forgoing Theory on Specific NESS Sys- 
tems 

In Sec. [3l we obtained the work and the heat for NESS systems described by a linear Langevin 
equation. In this section, first, using the simple models introduced in Sees. 12.11 and 12.21 as 
Classes A and B, we check that our formal expressions for the work and the heat in Sec. |3] 
indeed yield the physical NESS work and heat to maintain a NESS. For these discussions, we 
mainly use the dragged Brownian particle and pendulum models and omit arguments for the 
corresponding electric circuit models since their results can be obtained by the correspondences 
in Table [Tibet ween the Brownian particle (and pendulum) models and the electric circuit models. 
After that, we will also discuss the physical NESS work for more complicated NESS models 
introduced in Sec. I2.3l as Class C. 



5.1 Class A 

First, we apply our NESS Onsager-Machlup theory to the models discussed in Sec. 12. 1[ catego- 
rized as Class A. 

For the electric field driven model in Class A, using the force Fi = q£ the Lagrangian (|2TI) is 
represented by £ = —{af3/4:)[xs — qS/a + {m/a)xs]'^. Using this Lagrangian we can calculate 
the entropy production rate S± by Eq. (|28] ), then the heat Q^. by Eq. (I42l) and the work W± by 
Eq. (|50l ) using the energy of Eq. (O. Alternatively, using the force Fi = qS with the external 
nonequilibrium parameter fi = S, the work can also be calculated directly by Eq. (|54l ) with 
Eqs. (|55l) . (l56l) . (l57l) and (l58l) . These calculations are straightforward, so we omit their details 
and only show their results in Tabled In this Table we exhibited especially the work W±, since 
the heat Q± is given by Q± = yV± — AE± [i.e. Eq. ([501 )1 in terms of the work W± and the 
energy difference AE± of Eq. ([491 ). 

In this model, the physical NESS work should be given by the work done by the external electric 
force qS. This work indeed appears as W+ = /^^ ds qSxg in the above calculation for the work 
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(|54|) . This physical NESS work is zero at equilibrium £ = 0. Using the average velocity 
{xg) = V = q£/a for this model in the NESS, the average work rate corresponding to this 
physical NESS work W+ is given by {q8)'^/a in the NESS, which is proportional to the square 
of the external nonequilibrium parameter £. Therefore, the average work, as well as the average 
heat and entropy production rate, are strictly positive in NESSs, as we required in the end of 
Sec. [341 

Although it is not the physical NESS work, we can still calculate the work >V_ for the electric 
field driven model. It is given by J^^ mXsV, i.e. the work to maintain a motion of the particle 
with an average velocity v{= q£/a) by the total force mxs{= q£ — axs + Cs)- I^i the NESS, 
the average of this work W_ is zero since then (xs) = 0, so that following our requirement for 
the average physical NESS work to be strictly positive in the NESS, this work W_ cannot be 
the physical NESS work. 



5.2 Class B 

We now discuss the work and the heat for Class B. For the dragged Brownian particle model of 
this class described by Eq. the mechanical force is given by Fi = —k{xs — vs) [cf. Eqs. ([H) 
and ([171) 1 and then the Lagrangian ([2T1) by £ = —{ajS / A)\xs + k{xs — vs) /a + (m/a)xs]^. For 
the driven torsion pendulum model [cf. Eq. ©I, the force Fi and the Lagrangian C can simply 
be obtained using Table [H from the corresponding Fi and C for the dragged Brownian particle. 
Then, in a similar fashion as in the previous section for Class A, we can obtain expressions for 
the heat Q± and the work >V±, etc. for these models, using the NESS Onsager-Machlup theory. 
These results are also summarized in Table [2l 

From the works W+ and W_ in Table [H the work >V_ gives the physical NESS work to sustain 
a NESS for these models. Actually, the average work rates for these physical NESS works 
in the NESS are given by av"^ and ^H/a for the dragged Brownian particle and the driven 
torsion pendulum, respectively, which are strictly positive and even functions of the external 
nonequilibrium parameter, as required for the physical NESS work. 

It is important to note that the physical NESS work in Class B is W_, different from Class A 
where the physical NESS work is given by W_|_. Moreover, even within the same Class B with 
the same form of the Langevin equation, the physical NESS work is different for the dragged 
Brownian particle model, where W_ = — ds k{xs — vs)v, and the driven torsion pendulum 

model, where W_ = ds AigOg- This difference in the physical NESS work is due to a 
difference of the external driving force in other words, the difference between Eqs. ^ and 
© for the internal energy of these two models. 

It may also be noted that there is no difference between the internal energies and E^ [cf. 
Eq. ([48l) and Table [2l for the driven torsion pendulum model (as in the electric field driven 
model in Class A), so that we can obtain both the works W+ and W_ for this model. On the 
other hand, there is no E^ for the dragged Brownian particle model because that energy does 
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Class 


Class A 


Class B 


Model 


Electric field 
driven model 


Dragged Brownian 

particle in the 
laboratory frame 


Dragged Brownian 
particle in the 
comoving frame 


pendulum 


Langevin 
equation, 
Figure, 
Section 


Eq. ©[©], 
Fig.[I]:a)[(b)], 
Secs.|2.1| 
and|5.l| 


Eq. ©[©I, 
Fig.[2l;a)[(b)], 
Secs.|2.2| 
and|5.2| 


Eq. dH]) 
in Sec. |5.3| 


Eq. (©[Col)], 
Fig.|2l:c)[(d)], 

OCCS. IZ.ZI 

and|5.4| 


Nonequi- 

librium 
parameter 


£ 


V 




Internal 
energy 


1 • 2 








Time- 
reversal 
operator 


h 




Forces 


Ft^ = 0, 

f 1 , —nF 


f[''} = -i%{xs-vs), 
/i- = 


7i- = 


F^} = 
-aOs + Ms, 
Fi°^ = 0, 


NESS 
work 


= Jlds qSxs 




>v_=>vi"^ = 

-jlds {Kys+mys)v 


= J^^ds MsOs 


Average 
NESS 
work rate 


Q 


av'^ > 




Non- 
NESS 
work 


= jl^ds mXsV 
(Zero average 
in NESS) 


No W+ 
[The energy does 
not satisfy Eq. 
(l5Dfor/+.] 


= -Jlds aijsV 
(Zero average in 
NESS) 


- f'dsMs 



Table 2: Expressions for the work W± in the NESS models described by Langevin equations in 
Sees. I2.1[|2.2l and l5.3l as Classes A and B. We also show the external nonequilibrium parameter 
fi, the internal energy E, the relevant time-reversal operator i±, the forces , f[°2 and fi± 
to be used to obtain the physical NESS work, and the (strictly positive) average physical NESS 
work rate. The energy, the external nonequilibrium parameter and the work in this Table are for 
Brownian particle (and pendulum) models, and the corresponding quantities for electric circuit 
models can be obtained using the correspondences in Table [T] 



not satisfy the time- symmetric condition (|5T1) for the energy using /+, leaving /_ as the only 
possible time-reversal operator, giving the correct physical results. Therefore, there is no W+ 
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in this model. 

Finally, as shown in Table [21 the physical NESS works for the models of Classes A and B in 
Sees. 12. II and [Z2l are of two types: the type (l55l) for the dragged Brownian particle model (Class 
B), and the type (l56l) for the electric field model (Class A) and the driven torsion pendulum 
model (Class B). In the next subsection 15.31 we discuss another case in which the physical 
NESS work is of the third type (1571 ) of work. 



5.3 Dragged Brownian Particle in a Comoving Frame 

The Langevin equation dH) describes the dynamics of a dragged Brownian particle in the lab- 
oratory frame, where Xs is the particle position in the laboratory frame. In this subsection we 
discuss this dragged Brownian particle in the comoving frame [|5] |6l. In that case, the phys- 
ical NESS work is given by a qualitatively different type of expression than discussed in the 
proceeding sections [5Jl and \5J2\ This provides an interesting illustration of the general theory 
discussed in Sec. [3l 

The spatial coordinate of the Brownian particle in the comoving frame for this system is 
given by 

ys = Xs- vs, (70) 
and its dynamics is expressed by the Langevin equation 

mys = -nys- av - ays + Cs- (71) 

Note that different from the Langevin equation (Hj) for the laboratory frame, Eq. (TtTI) does not 
have an explicit time-dependence and the nonequilibrium effect appears just as a constant term 
—av. In this frame the internal energy E of the particle is given by 

E{ys,ys) = ^my^s + ^i^Vs (72) 

which is independent of the external nonequilibrium parameter fi = v, different from the lab- 
oratory case. Note that the internal energy (1721) for the comoving frame is different from the 
internal energy ([S]) for the laboratory frame because of a frame-dependence of the kinetic energy 

m. 

In this model, the mechanical force and the Lagrangian are given by Fi = —nys — av and C = 
— {a/3/A)[ys + {K,/a)ys+v + {m/a)ys]'^, respectively Using this force or Lagrangian, we can 
calculate the quantities like S±, Q±, >V±, etc., in a similar way as in the previous subsections 
l5.1l and [S!2l for the models of Classes A and B in the laboratory frame. We summarize the results 
in the 4-th column of Tabled 

Different from the models in Sees. l2.1l and [Z2l the physical NESS work W_ for this model in 
the comoving frame is of the type (|57l) involving the odd part force F^^^ = —av of the force 
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Fi. Note that this physical NESS work is obtained by using the same time-reversal operator 
/_ as in the laboratory frame, but that it includes an additional effect due to the d'Alembert 
type of force —myg, absent in the laboratory frame. This d'Alembert type force has no effect 
on the average physical NESS work nor on the average work rate av"^ in the NESS, which are 
therefore frame-independent. However, as discussed in Ref. f6\, fluctuation properties of the 
work are influenced by this d'Alembert type force. 

Another difference between the comoving and the laboratory frames in the dragged Brownian 
particle model is that in the comoving frame, the energy is time-reversal invariant satisfying the 
condition (ISTI) under both time-reversal procedures and Therefore, different from the 
laboratory frame, we can obtain the other work W+, which is the work to overcome the friction, 
i.e. W+ = — ds aygV^ The average of this work W+ is zero in the NESS, so that it cannot 
be the physical NESS work. 



5.4 Class C 



As the last example for nonequilibrium work, we consider stochastic models with two random 
noises as in Class C introduced in Sec. 12.31 One of these models is an example in which the 
work is given by the type (l58l) . 



For Class C, whose Langevin equation is expressed by Eq. (|T5|) . the mechanical force Fj is 
given by 



Fj{-Ks, S; T]) = T5j2 - K, {Xjs - Xks 



(73) 



for j ^ k, j = 1,2 and k = 1,2. Inserting Eqs. (l73l) and N = 2 into Eq. (|2T|) . the Lagrangian 
£ is given explicitly by 



K {Xis - X2s) , nil .. 
Xis H \ Xu 

«1 



n 2 



"2/52 



K {X2s - Xis) - r 7712 .. 

X2s H \ X2s 

0L2 0L2 



(74) 



as the sum of two terms due to the presence of two independent random noises. Applying our 
general theory given in Sec. [3] to this model expressed by the Lagrangian (|74|) . we obtain the 
quantities like 5"^, Q-t, >V±, etc. Especially, the work W±({xs}, /i) is given by 

AT /■* 



+ ( 1 + ^] r /" dsX2s^O 



AT 



T 



(75) 



^In Ref. the work — ds aijsV was called the "energy loss by friction". 
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and 



W_(|xJ,/i) = / ds ' 



4T 



\m2X2s + (a;2s - xxs)f + (a2a;2s - T) 



«2 

+r / ^— + C 

-t "2 



AT 



T 



(76) 



up to first order in | AT/T], respectively, with AT = Ti - T2 = 2ATi = -2AT2 and T = T. 
The works (|75]) and (1761) are zero at equilibrium where AT = F = 0. 



Eqs. (1751) and (l76l) gives the works in a unified form for the energy transfer model driven by a 
temperature difference and the electric circuit with two resistors in Class C. In the remainder of 
this section, we discuss their physical meanings in these two models separately. 



5.4.1 Energy Transfer by a Temperature Difference 



We first discuss the energy transfer driven by a temperature difference, which is described by 
the Langevin equation (fTTI) . i.e. Eq. (fT5l) in the case of mi = m2 = m, ai = 02 = a and 
r = 0. In Table. [3] we show the work >V± obtained from Eqs. (1751 ) and (|76l ) in this case. These 
works are typical examples for the work as given by Eq. (l58l) for N = 2. 

To choose the physical NESS work from W+ and W_ in Table. [3]it is enough to note that in the 
NESS the physical NESS work should be zero in the case of k = 0, i.e. no coupling between the 
two reservoirs. The work >V_ does not vanish in the case of k = 0, while the work W+ vanishes 
in such a case apart from a boundary term, which disappears in the NESS on average. Therefore, 
we conclude that the work W+ obtained by the time-reversal operator J_|_ gives the physical 
NESS work in the energy transfer driven by a temperature difference. As further evidence for 

the appropriateness of the physical NESS work W+, we note that the average work rate W+ 

corresponding to this work W+ in the NESS is given by W+ = anksAT'^ /[2T{a'^ + mK)] up 

to the second order in AT, as shown in Appendix lC.il Thus, W+ is strictly positive and an even 
function of the temperature difference AT, which are necessary conditions for the average work 

rate to keep the system in a NESS. This average work rate W+ has some interesting features 
due to the term mn, which includes inertia. Its value is zero at a = (as well as at k = 0), 
has a finite maximum value [W+ = ksAT"^ (4TA/m) at a = \JmK as a function of «, and 

W+ akB^T"^ / {2'mT) for k +00 as a function of k], and is close to its over-damped 
value (at m = 0) for large a or small k. 
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Class 



Model 



Class C 



Energy transfer model 
driven by a temperature difference 



Electric circuit 
with two resisters 



Langevin 
equation, 
Figure, 
Section 



Eq. O, Fig. da), 
Secs.Oand lSATl 



Eq. m, Fig.db), 
Sees. O and [5A21 



Nonequi- 

librium 
parameter 



AT 



Internal 
energy 



E 



1^^2 I 1 



j=l 2 



mx 



js "I" 2^ ("^1* ^2s) 



1 r -2 I {qis-q2sf 

o^^ls H 



2C 



Time- 
reversal 
operator 



Forces 



Fif = FiJ = 0, 

/l+ = /2+ = 



1+ — ^cvyis l^s). 



(o) 



/i+ = 0, f: 



2+ 



2+ 



0, 



NESS 
work 



AT 
2T 



1 ( ■ 2 '2"^ 1 ( ■ 2 '2"^ 



Average 
NESS 
work rate 



2T(a2+mK) 



> 



R1+R2 



> 



Non- 
NESS 
work 



25/1 



+ 2k {xu - X2s) 



m[Xis - X2s> 



Table 3: Expressions for the work W± in the NESS models of Class C. We also show the 
external nonequilibrium parameter /x, the internal energy E, the relevant time-reversal operator 
J± and the forces Fj^, f'^°2 and fj± to be used to obtain the physical NESS work, and the 
(strictly positive) average physical NESS work rate. 
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5.4.2 Electric Circuit with Two Resistors 



We categorized the electric circuit with two resistors, described by the Langevin equation (fT3l) . 
in the same Class C as well as the above energy transfer model, although they look very different 
at first sight. The common feature for these models are that both systems are coupled to two 
independent random noises. However, different from the electric circuit models in Classes A 
and B, results for this electric circuit model in Class C cannot be derived from the ones for 
the corresponding Brownian model simply by using the correspondences in Table [U since the 
Langevin equations (fTTl) and ( fT3] ) for the models in Class C have different forms. Therefore, we 
have to discuss physical quantities of this electric circuit model in Class C separately. 

The electric circuit model with two resistors is described by the Langevin equation (fT5] ) in the 
case of mi = L, m2 = 0, F = V, k = 1/C, aj = Rj, xjs = qjs and AT = 0. Therefore, 
from Eqs. (l75l) and (|76|) the works W± for this system can be obtained (cf. Table [3]). It is clear 
that the work W+ is the physical NESS work done by the battery with the voltage V to produce 
electric current = q2s- We note that the physical NESS work W4. for this model is of the 
type (|56l) , different from the energy transfer model in which the physical NESS work is of the 

type (l58l) . We also show in Appendix IC.2I that in this model the average work rate W+ in the 

NESS is given by W+ = V'^/{Ri + R2). This average work rate is strictly positive and is an 
even function of the external nonequilibrium parameter V. 



6 Summary and Remarks 



In this paper we have discussed a method to calculate the work done on and the heat removed 
from a system to maintain it in a NESS. This was based on a NESS Onsager-Machlup theory 
for stochastic systems with Gaussian-white random noises. The work and the heat to maintain 
a NESS appear only for NESSs, and not for equilibrium states. In our approach we obtained 
the heat as the time-irreversible part of the probability functional for paths in a functional space 
via a Lagrangian, and from it the work, using the energy conservation law. We incorporated 
multiple possibilities for the time-reversal procedure for NESSs, due to the external nonequi- 
librium parameters to specify the NESS. We also indicated that the separation of the heat into 
work and an energy differences is not unique, so that we can get different expressions for the 
work for systems described by the same dynamical equation. We showed that the work can 
consist of four parts: one coming from a partial derivative of the energy with respect to time, 
the second one due to an external driving force, the third one caused by a time-irreversible force 
and the last one due to temperature differences of reservoirs. We also derived nonequilibrium 
generalizations of the detailed balance condition, leading to transient fluctuation theorems for 
the work distribution functions. Our theory was illustrated by various NESS models, for exam- 
ple, dragged Brownian models, electric current models, an energy transfer model driven by a 
temperature difference, demonstrating the above four kinds of components for the work. 

Finally, we make some remarks on the contents of this paper. 
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[1] We first make some remarks about the ambiguities to define the physical NESS work by the 
NESS Onsager-Machlup theory. 

la) The first ambiguity is a non-uniqueness of time-reversal procedures due to the presence of 
external parameters, defining the NESS. This can be treated by the introduction of two time- 
reversal operators /+ and leading to two candidates for the physical NESS work, i.e., W+ 
and W_. To chose the physical NESS work, i.e. the actual work to maintain the system in a 
NESS, from W+ and W- we had to use model-dependent physical arguments, for example, (i) 
The strict positivity of the average work in the NESS [cf. the electric field model (Class A) and 
the dragged Brownian particle model in the comoving frame (Class B)], since positive work has 
to be done to sustain the system in a NESS, (ii) The time-reversal symmetry (ISTI) for the internal 
energ)!^ [cf. the dragged Brownian particle model in the laboratory frame (Class B)]. However, 
a general criterion to choose the physical NESS work from W+ and W- is an open problem. 

lb) The second ambiguity to choose the physical NESS work is due to multiple possibility 
to separate heat into work and an energy difference. In our theory, this ambiguity appears in 
Eq. (|47l) . i.e. when the even part F^'^^ of the external force is separated into the two terms 
—dU±/dxjs and fj± as an essential step to define the energy E± by Eq. (|48]) . Note that like 
the first ambiguity discussed in the previous paragraph, this ambiguity also does not appear at 
equilibrium because there is then no external driving force fj±. We demonstrated this second 
ambiguity concretely using the dragged Brownian particle model and the driven torsion pen- 
dulum model, which are described by the same Langevin equation but have a different form 
for their internal energies. In fact, in the driven torsion pendulum model, the torque Ms{= ^s) 
can be regarded as an external force not part of the system and not contributing to its internal 
energy, while in the dragged Brownian particle model the force kvs, which corresponds to Ms, 
does contribute to the internal energy and is therefore regarded to be as part of the system. In 
general, this ambiguity can be resolved only on physical grounds, i.e. by considering what is 
physically an internal energy for each nonequilibrium model|^ In another example, in the elec- 
tric field model in Sec. 12. 1[ we took the internal energy as not to include the force qS, since this 
force was regarded as an external force leading to the physical NESS work W+ = 
shown in Tabled However, purely mathematically, we could have chosen the internal energy 
as (l/2)mx^ + qSxs [cf. Eq. ©J. This choice of energy leads to the work W+ = 0, which is 
unphysical, since positive work must be done to keep the system in a NESS. 

Ic) An additional remark related to the above points is that the energy conservation law and the 
second law of thermodynamics are not sufficient to determine the physical NESS work W and 
heat Q uniquely, because one can always add the same functional X of a path with a positive 
functional average (( X ))^ > 0) to both the work and the heat so that these new "work" W + X 
and "heat" Q + X satisfy the energy conservation law Q + X = ( W + X) — AE and the second 
law of thermodynamics ( Q + X )^ > 0. However, in our NESS Onsager-Machlup theory this 

^ Note that if the internal energy were time-irreversible, then the energy of the final state of the forward path 
would not equal the initial energy of the backward path. 

This difficulty does not occur if one consider only the work ( fSSt given by a partial time-derivative of the 
energy, as done in Refs. 1271 l28l . However, as shown in this paper, this relation connecting the work and the 
energy is not valid in general since the work can have other contributions (|56] |. dSTl i and dSSl l. 
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ambiguity does not occur, because the heat is fixed by Eq. (|46|) via the probability functional 
V^{{xs};tJ,) of paths. 



[2] We now make some remarks on the time-reversal operator i± for our NESS Onsager- 
Machlup theory. In particular, we comment on the relation of these operators with other possible 
time-reversal operators which have been used in the literature. 

2a) In Ref. ||30ll a time-reversal operator /' is introduced by 

P J dsY{xs,^s,Xs,s) = J dsY(xs,-±s,yis,s), ill) 

for any function ^ (x^, x^, x^, s) of Xg, x^, x^, s, under which the explicit time-dependence s 
in F(xs, Xs, Xs, s) does not change its sign. By the definition (1771) . the time-reversal opera- 
tor /' is independent of the external nonequilibrium parameter /i. This time-reversal operator 
/' is different from the operator I± used in this paper, but can yet be considered as a special 
case of the time-reversal operator This, because Eq. (|25] ) for the operator /„ becomes 
Eq. (1771 ) if the explicit /x- and s-dependences of F(xs5 s] /i) appear only via ^is, i.e. if 

y(xs, Xs, Xs, s; /x) = F(xs, x^, x^, /xs) with a function F(xs, x^, x^, /xs). However, the opera- 
tor /' can not be used as an operator on a general functional X({xs}), contrary to the operator 
I± defined by Eq. ((22)) . Moreover, the time-reversal operator I' fails to produce the correct 
physical NESS work for some nonequilibrium models discussed in Sec. 15.31 For example, if 
we were to apply /' to the dragged Brownian particle model in the comoving frame, then we 
would obtain the work to overcome the friction only, i.e. )/V+ = — ds aijsV, instead of the 

physical NESS work — J^^ ds {ki/s + mys)v for this model. For these reasons we did not use 
the time-reversal operator (1771) in this paper. 

2b) As another example, Ref. [31] considered the case in which the sign of the external nonequi- 
librium parameter does not change in a time-reversal procedure. This case corresponds to the 
time-reversal operator /+ in this paper. However, this operator does not always produce the 
correct physical NESS work for some models, for example, for the models of Class B in Sees. 
15.21 and 15.31 Therefore, this operator is not general enough to construct the physical NESS 
work and heat based on the NESS Onsager-Machlup theory for a sufficiently general class of 
nonequilibrium systems which include all NESS models discussed in this paper. 

[3] In this point we make some remarks on the transient and the asymptotic fluctuation theorems 
in the context of the NESS Onsager-Machlup theory. 

3a) We first consider the transient fluctuation theorem [[TOll . We will argue that the transient 
fluctuation theorem can be derived purely formally, as a mathematical identity, without any 
specifications of the dynamics of the system. To show this, we first define, purely formally, 
without any physical interpretation, the functional Q({xs}) by: 

where P({xs}) is the probability functional of the path {xs}se[to,t]- Here, the operator JT" is a 
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general time-reversal operator satisfying the condition j'^ = \. Next we introduce again purely 
formally another functional W({xs}) defined by: 

W({xJ) = Q({xJ) + A^ (79) 

where l\E = E{xt, xt, t) — -E(xj(,, x^^, to) is a boundary term as the energy difference between 
the energies of the system under consideration at the final time t and the initial time to- Inserting 
then (|79l ) into Eq. (fTSl) . we obtain identically: 

e-^[^«^=»-^^]p({xJ)f?(i,„,x,„,to) = £)(it,x„t)iP({xJ), (80) 

where ^(x^, x^, s) = exp[— /5 E{±s, x^, s)] is a canonical-like distribution, with the formal 
partition function Zs = J dxg f dig exp[— /3£'(xs, Xg, s)]. Here AjF = Tt — J^to with jFg = 
-[3~^ In Zs, a formal free energy-like quantity. If then JW = -W [cf. Eq. ([53]) for J = /±] 
then the work distribution, Pw{W, t) = { 6 (W — W({xs})) satisfies formally a generalized 
transient fluctuation theorem similar in the form Eq. (|69l ): 

iP^(-iy,t) 

for the canonical-like initial condition f?(xtp , x^p , to) • However, the derivation of Eq. (|8T1) shows 
that if one introduces formally any quantities Q and W defined by the equations (l78l) and (|79l) . 
respectively, then the Eq. (ISTI) follows as an identity. To the contrary as shown in this paper 
the NESS Onsager-Machlup theory does define physical quantities Q({xs}) and W({xs}) [cf. 
Eqs. (|46l) . (l50l) and (l53l)1. which lead to a transient fluctuation theorem of the form (l69l) . 



3b) In this paper we derived transient fluctuation theorems using the NESS Onsager-Machlup 
theory for the work distribution P^±(iy, t; fx). The transient fluctuation theorems lITOl (as well 
as those in Refs. Il27ll29l ) hold for any time for an equilibrium initial distribution function. For 
a general, i.e. any, initial condition (including equilibrium), an asymptotic fluctuation theorem 
lfT2l can be derived, in the form: 

Hm -^4^ = e^^. (82) 

for a work distribution function P^(VF, t). In spite of its analogy with Eq. (ISTI) . it is of an 
entirely different nature lfTTl[32l . To be sure, Eq. (|82l ) can be formally derived in the long time 
limit from Eq. dSB, if Pw{W, t) = JP^{W, t), AjF = and the initial condition is given by 
^)(xt(,, xjq, to). However, Eq.(f82l) makes a much stronger statement, because it not only holds 
for the equilibrium initial condition, but for any initial condition, which would require some 
dynamical stability condition, e.g. a condition for the system to approach a unique steady state 
for t — > +00. 



In this connection, one can ask whether both work distribution functions Pw+{W,t; /i) and 
Pw-iW, t; /x), given by Eq. (|63l) . obey the asymptotic fluctuation theorem for any initial con- 
dition. The answer is no, as was shown in Ref. |l5l for the distribution function Pw+{W, t; /x) 
for the dragged Brownian particle. In fact, this could occur when the work distribution function 
Pu,{W, t) appears as a boundary term of the form X(xt) — X{-Kt^^), for a function X{yis) of x^, 
rather than as a functional along the full path {xsjsgj^Q tj. 
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A Work based on the NESS Onsager-Machlup theory 



In this Appendix we give a derivation of Eq. (l54l) . 
Using Eqs. gS]), gV]), (SHI), (ill) and (I50l) we obtain 



W±({xJ;/.) + 5^- / ds 



N 



i=i 



T j.t 



F^2i^s, s; rj) - rrijX 



X 



3S 



a 



N 



2 T 



t 

X I ds 



1 rrij 
Xjs H Fj{xs, -s; ±r]) x 



1 2 



O 



AT, 



T 



N 



9f/±(x,,s;?7) 



dxjg 



fj±{^s,s;r]) -mx 



Xjs + AE± 



ds 



9f/±(xs,s;?7) 



N 



ds 



Xs, S, Tl)Xjg 



ds 



^ d fl ^ dU±{xs,s;'n) . 9f/±(x^,s;j7) 



Lj=i 



9x., 



ds 



ds 



+AE. 



dE±{±s,yis,s;r]) 
ds 



N 



+ '^fj±{y^s,S]r])xjs 



± 



-t 



ds 



ds 



^E±{i^s,:>^s, s; ri) 
ds 



N 



^fj±{^s,s;r])x 



(A.l) 



Therefore, we obtain Eq. ^ with Eqs. ([55]), §7} and (l58l) . 
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B Transient Fluctuation Theorems 



In this Appendix we derive Eqs. (1681) and (1691 ) in the case of the initial condition (|67I) . 

Using the definition (l66l) of £w±io; t] A*) with the functional average (|64|) under the initial dis- 
tribution function (|67l) we obtain 

_ f f r('<'t,Xt) = (xf,Xf) r- p 



(x_t,X_t) = (Xi,Xi) 

xe---'^)^±«''^>^/^)p.({x4)f5±(i,,x„-t;r7) 



e-^^-^± / c/x/ / c/i/ / Dx, / c/x,; / rfii 

^ J ^(x_t,X_t) = {Xi,Xi) ^ 

xe'^^±«''^>^/^)f5±(i/,x;,t;r?)4Px({x.};/.) 

//•(xt,xt)=(x/,x/) 
d±j / Dx, / C/X,; / d±i 

J (x_i,X_t) = {Xi,Xi) 



X 



Q^{-±f, X;, -t; ±r,)4e-'^^±«''^>^/^)P.({x,}; /x) 



(x_t,x_t)={x/,xj) 



= e"^'^-^±/± / dx/ / d±f I Px, / dxi / rfxi 

^ ^ J(xt,Xt) = (Xi,Xi) J J 

Xf,±(-i^, x;, -t; r7)e-'^^±«^»>^'^)p.({x,}; /x) (B.l) 
- ^ r r /•(xt,xt)={x^,i/) /• 
= e-^^^±/± / dx/ / dx/ / Px, / dxi / d±i 

J J ^(x_t,X_t) = {Xi,Xi) ^ J 

_ xe"'^^±«^^>^/^)p,({xJ;AA)f?±(i„x„-t;r,) (B.2) 
= e-^^^±4^«,±(a,t;/x) (B.3) 

where we used Eqs. (I22l) . (1531 ). (|591 ) and g±{±f,Xf,t;T]) = g±{—±f,:x.f,—t;±r]) noting 
Eq. (ISTI) . Here, in the transformation from Eq. (IB.ll) to Eq. (IB .21) we exchanged the in- 
tegral variables Xi,Xj,xj and xj with xj,xj,Xj and Xj, respectively, and used the relation 
g±{^i, Xj, —t; T]) = f?±(— Xi, Xi, — t; 77). Therefore, we obtain Eq. (l68l) . 

Using Eq. (|68l) we obtain 

= e-^^^^i±S.^±(p-a,t;ti). (B.4) 
From Eqs. (|65l) and (|B.4I) we can derive 

P^±(iy,t;/x) = — / rfae*'^^e-^^^±4^«,±(/5-^a,t;/x) 



_ T /•+00— fi/3 

J~oo-if3 

e^(^-^^^)i^P^^(-W,t;f^) (B.5) 
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where we used a' = —1(3 — a and the fact that noting Eq. (l66l) the function £w±{io'',t; fi) 
^-la'w (jQgg j^Qj \\dMe, any pole in the complex plane for Im{cr'} G [0, —(3] with the imaginary 
part Im{cr'} of a'. Eq. (IE31) leads to Eq. (1^91) . 



C Average Work Rates in Class C 



C.l Energy Transfer Model by a Temperature Difference 



In this Appendix, we calculate the average work rate for an energy transfer model driven 
by a temperature difference in a NESS. 

First we introduce Xt and Axt as 

_ Xit + X2t . . 

Xt = , (C.l) 

Axt = xit - X2t- (C.2) 

From Eqs. (fTTI) . (IC.ll) and (IC.2|) we can derive the Langevin equations for xt and Axt separately 
as 

mxt + axt = e ™ — e — — = , (C.3) 

dt dt 2 ' ^ ^ 

mAxt + aAxt + ^nAxt = e-'-'^^+'^'^^-^e'^''^^e'^'^Axt = Cit - C2t (C.4) 

dt dt 

with Xt = d'^Xt/dt'^ Xt = dxt/dt, Axt = d'^Axt/dt'^ and Axt = dAxt/dt. Here, Um., uja and cui, 
are defined by = a/m,uja = a/w^ — {8k /m) and Ub = (u;m, — u;a)/2, respectively. Solving 
Eqs. (|C.3I) and (IC.4I) we obtain 

Xt = Xt, + — [1 - e-'^^-^'-''^ tto 

+— / dui / rfn2 e-^-("^-"^) (Ci., + C2«J , (C.5) 

-^"^ J to J to 

Axt = |Axi„ + -^[l-e--"(*-*»)]KAxi„+Ai;ij|e--''(*-*'') 

+ 1 f du, r dU2 e-^a{u,-U,)-.,{t-U,) (^^^^ _ ^^^J _ ((..6) 

Using Eq. (IC.5I) we obtain 

= e--'"^*-*")^;^^ + ^ [ du e-'"^*-") (Ci. + C2.) • (C.7) 



2m 

for the time-derivative of the position xt. 



to 
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By the expression of the work W+ in Table. [3]and using Eqs. (IC.ll) and (IC.2I) the average of the 
work rate can be expressed as 



'rfW+({xJ,AT) 
dt 



AT 



K (AxtXt) 



m d {AifXt) 
2" Jt 



AT 



T 



(C.8) 



using Axt and Xt for the energy transfer model driven by a temperature difference. In the 
NESS, the quantity {AxtXt) should be independent of time t, so we can neglect the term (m/2) 
d{AxtXt) / dt in the right-hand side of Eq. (IC.8I ) in such a state. Moreover, the NESS should be 
realized in the long-time limit, so we can neglect the first terms of the right-hand side of Eqs. 
(IC.6I) and (|C.7I) to calculate a quantity in the NESS. Using these points, the average work rate 
W+ in the NESS up to the second order of AT^ can be calculated by 



1. ^^^T ,^ 

lim -—- {AxtXt) 

t— »+oo J 

lim — - / dui 



duo 



dUs ((Clna + C2«2)(Cl«3 + (2^3)) 



xe 



-U)a{ui—U2)—UJb{s — U2)~iOm{s — U3) 

anksAT^ 



rn?T{uJa + UJb + ^m){^b + t^m) 

ankeAT^ 



rriK] 



(C.9) 



where we used the relation {{ds + C2s)(Ci« ~ C2u)) = 2akBAT6{s — u). 



C.2 Electric Circuit with Two Resistors 

In this Appendix, we calculate the average work rate W+ for the electric circuit model with two 
resistors. By the expression shown in Table [3] for the work W+ in the electric circuit with two 
Resistors, the average work rate is given by {Vl2t) = V{q2t), so that calculation of {q2t) in the 
long time limit is sufficient to calculate the average work rate W+ in the NESS. 

To calculate {q2t) we first note that 

C iL{q,t) + R,{qu))) = fet) - M (CIO) 
C{V-R2{q2t)) = {q2t)-{qit) (C.ll) 

by taking the ensemble average of the Langevin equation (fT3l) . Inserting (qu) = {q2t) + 
C{R2{q2t) - V) from Eq. (ICTTl) into Eq. (ICTOl) we obtain 

LCR2^ + {L + CRM ^ + {Ri + R2) 4>t = (C.12) 
dt-^ dt 

for (pt = {q2t) — y/{Ri + -^2)- Note that the differential equation (IC.12I) for (j)t has the same 
form as the one for {Axt) obtained by taking the ensemble average of the Langevin equation 
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(IC.4I) . whose solution satisfies lim^^+oo ( Axf) = by the average of Eq. (IC.6I) . In a similar way 
for {Axt) we can also show that lim^^+oo 0t = 0, i.e. 

which is the value {q2t) in the NESS. Using this and W+ = limt_»+oo V{q2t), we obtain W+ = 
Vy{Ri + R2). 
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